Pivot Vector Space Approach for Audio-Video Mixing

artist usually adds the musical accompaniment to video. Employing such artists is expensive and not feasible for a home video presentation. Our automatic audio–video mixing technique is suited for home videos. It uses a pivot vector space mapping method that matches video shots with music segments based on aesthetic cinematographic heuristics. A udio mixing is an important aspect of cinematography. Most videos such as movies and sitcoms have several segments devoid of any speech. Adding carefully chosen music to such segments conveys emotions such as joy, tension, or melancholy. It also acts as a mechanism to bridge scenes and can add to the heightened sense of excitement in a car chase or reflect the somber mood of a tragic situation. In a typical professional video production, skilled audio-mixing artists aesthetically add appropriate audio to the given video shots. This process is tedious, time-consuming, and expensive. With the rapid proliferation in the use of digital video camcorders, amateur video enthusiasts are producing a huge amount of home video footage. Many home video users would like to make their videos appear like professional productions before they share it with family and friends. To meet this demand, companies such as Muvee Technologies (http://www.muvee.com) produce software tools to give home videos a professional look. Our work is motivated by similar goals. The software tool available from Muvee lets a user choose a video segment, audio clip, and mixing style (for example, music video or slow romantic). The Muvee software automatically sets the chosen video to the given audio clip incorporating special effects like gradual transitions , the type of which depends on the chosen style. If a user chooses an appropriate audio and style for the video, the result is indeed impressive. However, a typical home video user would lack the high skill level of a professional audio mixer needed to choose the right audio clip for a given video. It's quite possible to choose an inappropriate audio clip (say the one with a fast tempo) for a video clip (one that's slow with hardly any motion). The result in such a case would certainly be less than desirable. Our aim is to approximately simulate the decision-making process of a professional audio mixer by employing the implicit aesthetic rules that professionals use. We have developed a novel technique that automatically picks the best audio clip (from the available database) to mix with a given …